All content

Traditional large language models (LLMs) are extraordinarily useful. They can summarise, draft, explain, search, translate, simplify and accelerate work that previously sat in queues, inboxes and clinical admin backlogs. But we need to be brutally clear about what they are. They are not truth machines. They are language machines. My friend Herb Roitblat’s critique goes straight to the root of the issue. LLMs predict likely words. They do not, in their traditional form, represent truth. Roitblat’s framing is that probability and reinforcement can guide which tokens are selected, but this is not the same as the system knowing whether a proposition is true. Reliath’s position is even more direct: the problem is structural because the unit of analysis is the token, not the fact.[1] That distinction matters everywhere. In healthcare, it matters more. A bad answer in marketing is embarrassing. A bad answer in healthcare can change a pathway, delay a diagnosis, distort a record, mislead a patient or create a false sense of clinical certainty. The real problem: fluent nonsense at the point of trust The danger with LLM hallucination is not simply that the model gets something wrong. People get things wrong all the time. The danger is that the model gets something wrong while sounding structured, fluent, balanced and authoritative. In healthcare, that is an especially toxic combination because patients often lack the knowledge to challenge the answer, and clinicians are already overloaded. This is why hallucination is not just a technical bug. It is a trust failure. The World Health Organization (WHO) has warned that large multimodal models used in health can produce false, inaccurate, biased or incomplete statements, and that this can harm people when used for health decisions. It also highlights automation bias, where clinicians or patients overlook errors because the system appears authoritative.[2] That is the strategic issue. Not whether AI can be useful. It clearly can. The issue is where we place it in the system, what level of authority we give it, and whether the output is grounded in verifiable facts or simply dressed in confident language. Why healthcare makes the hallucination problem worse Healthcare is not a clean data environment. It is full of abbreviations, conflicting notes, outdated pathways, local protocols, missing observations, patient-specific exceptions and subtle clinical context. A word like “negative” can be life-changing depending on where it sits. A missing allergy can be catastrophic. A fabricated instruction in a discharge summary can move from screen to ward to patient before anyone has noticed. Recent research into LLM-generated clinical notes found a 1.47% hallucination rate and a 3.45% omission rate across clinician-annotated sentences. That sounds low until you realise that 44% of hallucinated sentences were judged major, meaning they could affect diagnosis or management if left uncorrected.[3] This is the healthcare problem in miniature. The percentages may look manageable. The consequences are not. Guardrails are not enough A lot of AI strategy today is built around mitigation: use better prompts, add retrieval, add a guardrail, add a human in the loop, add a second model to check the first one. All of these can help. None of them changes the fundamental nature of a traditional LLM. Herb’s challenge to the market is that guardrails often mask the problem rather than remove it. RAG can improve grounding, but it is still vulnerable to retrieval errors, source errors, chunking errors, interpretation errors and confident synthesis of the wrong material. Herb instead argues for shifting from tokens to factoids and facts, with “Truth Profiles” and logical or semantic representations designed to distinguish verified information from hypothesis or fabrication. That is an important strategic shift. The goal is not better autocomplete. The goal is accountable intelligence. What this means for AI in healthcare Healthcare AI cannot just be plausible. It has to be auditable. It must show what it knows, where it got it from, what is uncertain, what is missing and what should not be inferred. That means future healthcare AI systems need to separate four things that traditional LLMs often blur together: known facts, clinical interpretation, hypothesis and recommended action. Mix those up and you create danger. Keep them separate and you create a system clinicians can inspect, challenge and use. If the system can only generate likely language, then it must be treated as an assistant. If it can represent propositions, provenance, uncertainty and truth values, it starts to become something closer to clinical infrastructure, subject of course to validation, regulation and real-world safety testing. The strategic takeaway AI will absolutely transform healthcare. But the winners will not be the organisations that adopt the most AI the fastest. They will be the organisations that understand where AI is safe, where it is dangerous, where it is merely impressive and where it is genuinely trustworthy. The next phase of healthcare AI cannot be built on beautiful answers that may or may not be true. It has to be built on verifiable facts, clear provenance, explicit uncertainty and clinical accountability. Because in healthcare, the question is not “can the AI answer?” The question is “can we trust what happens next?” References Roitblat H. The self-curation challenge for the future of AI. 9 March 2025. WHO. WHO releases AI ethics and governance guidance for large multi-modal models. World Health Organization, 18 January 2024. Asgari E, Montaña-Brown N, Dubois M, et al. A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation. NPJ Digital Medicine, 2025; 8 (274). Further blogs from Richard: The harsh interface between patient care and automation led to a highly avoidable death AI found to not speed up lung cancer diagnosis—AI alone is not enough

This account was shared with by Louise, a patient who recently underwent a hysteroscopy in Liverpool. Having read about many negative experiences, she wanted to share her positive and painless experience, highlighting the need for less variation and better experiences for all women.  I was saddened to read the horrendous experiences of hysteroscopy shared by women on this site and only discovered the accounts shared here after my procedure yesterday. I am 54 and underwent a hysteroscopy including biopsies (polyp) and a Mirena fitting after experiencing irregular perimenopausal bleeding. I have one child who was delivered by emergency C-Section 12 years ago (which was truly horrific and is a whole other topic and hospital). My hysteroscopy procedure took place at Liverpool Women’s Hospital and I can only describe the whole experience as gold standard. I was seen and treated under the 2-week target for referrals. When I arrived the gynaecology outpatient department was calm, relaxing and well-staffed by very caring and attentive nurses. I was offered a Saturday morning appointment that meant no time off work, which was a plus for me. Each stage of admission, observations, pain relief, preparation and the actual procedure was carried out with care and absolutely everything was explained to me in great detail. I was made aware during my referral appointment a few days previously about the possibility and levels of pain to expect and the pain relief options that would be available. I was expecting the worst, although I did feel mentally prepared. Pre-op I was offered ibuprofen and a diclofenac suppository which I took and during the procedure I had a local anaesthetic and plenty of gas and air. The local anaesthetic was akin to how it feels at the dentist, weird but I found it bearable. I was welcomed into the theatre by a very friendly female consultant and three nurses who explained everything and put me totally at ease. My experience of the actual procedure and how it felt was a level up from the initial speculum exam at referral stage and it felt scratchy and pinchy rather than painful, saline was introduced to help with the imaging (which I chose not to watch on the screen but could have if I wanted to). The gas and air helped massively (I took so much it made me burst out laughing and the team were worried I was crying!). It was over very quickly - I would say 15 minutes. The consultant asked me when I was dressing how my pain was on a scale of 0-10 and I answered honestly that it was 0. I experienced no bleeding at all. I was then asked to relax on a recliner in recovery with tea and biscuits and I was monitored for approximately 30 minutes. From arrival at 8.00am to discharge at 10.30am I felt looked after, cared for, respected and treated with the utmost dignity and compassion. I didn't need to stay in recovery long as I felt so well but I feel they wouldn't have discharged me in a hurry if there had been any signs of anything worrying. I was a bit anxious how I would feel after the diclofenac wore off (which I was advised was 16 hours) but I have woken up this morning with no pain, no bleeding and just a sense of how fortunate I have been to be treated at Liverpool Women’s Hospital. It really concerns me to hear of so many awful experiences around the country and just wanted to highlight for balance how well the experience went for me. I am lucky to live in Liverpool, and my heart goes out to those women who have been traumatised. Care shouldn't vary depending on where you are treated. Thank you for reading my story.

Julie Storr is an expert in the field of patient safety, quality and infection prevention and control (IPC), and a Topic leader for the hub.  In this blog, Julie explores how IPC guidance can inadvertently lead to psychological harm, when it is not applied through a person-centred lens. Drawing on literature and reflecting on the COVID-19 pandemic, she questions whether guidance supports both compassion and safety when applied in practice.  Infection prevention and control guidance Healthcare guidance is designed to support the delivery of safe, high quality care by providing clear and consistent recommended best practices based on available evidence. In infection prevention and control (IPC), guidance typically focuses on: transmission routes precautions clinical interventions isolation and a range of environmental controls. At the point of care, guidance is interpreted by health care workers, in context and often under pressure. IPC is widely described as fundamental to patient safety and quality of care. Its foundations in microbiology and epidemiology have saved countless lives and it has deep roots in the biomedical model. From Louis Pasteur to Florence Nightingale, its legacy is grounded in germ theory, surveillance, and control. This has resulted in highly effective, evidence-based systems and has also shaped guidance that is often highly technical, precaution-focused, and written in absolutes. The cost to human connection During COVID-19 the gap between guidance and practice was brought to the forefront. Preventing the transmission of a single infection became the dominant goal, sometimes at the expense of human connection. Stories of people dying without loved ones, or families waving through windows, are reminders that safety without humanity can equally result in harm. This gap between policy and practice became highly visible and caught the attention of many working in the field of IPC and beyond, including myself. Elements of this story have been told many times now, including in a previous blog for Patient Safety Learning. When ‘technically safe’ risks psychologically harm In reviewing a sample of international and national IPC guidelines recently, a few things stood out. Many run to hundreds of pages, detailing precautions in depth. The language is directive, one guideline from the English NHS used the word “must” close to 100 times, reinforcing certainty and compliance. By contrast, words like person-centred, compassion, loneliness, or humanity were rarely, if ever, used. Even anxiety appears only occasionally, often without guidance on how to allay fear. What is less visible in much IPC guidance is the person. This has been described by colleagues as the grey space, where the human dimension is under-specified and staff are left to navigate complexity themselves often under pressure. In those conditions, guidance can quickly become black and white. This can lead to decisions that are technically safe but that may contribute to psychological harms. As one reflection from the period of the pandemic put it, in the case of some loved ones “we protected them to death.” The implementation gap This tension between IPC guidance as written and as experienced in practice reflects a wider challenge, long recognised in patient safety: the implementation gap. As highlighted by Patient Safety Learning, this is the disconnect between patient safety guidance working in theory, but not in practice. Actions that may appear to address patient safety issues failing to account for a wide variety of organisational context, culture and capacities. In the context of IPC, guidance may be evidence-based and technically robust, but if it does not account for the realities of care delivery or the human needs of patients, it risks being applied in ways that are rigid, inconsistent or inadvertently harmful. Patient Safety Learning have emphasised that this gap persists where there is a lack of joined-up approaches, weak systems for sharing learning, limited oversight, and unclear leadership. Building on existing learning There are examples to learn from. Some guidance now acknowledges the psychological impact of isolation on people and a recent IPC guideline from Ireland is strong on person centredness. Frameworks from America and Canada have been developed that support ethical decision-making. But certainly within the guidelines I reviewed, in the majority, these elements are often brief, dare I say tokenistic, a sentence or at most a paragraph in documents of hundreds of pages, rather than embedded. Patient safety literature is increasingly recognising the importance of compassion as a core mechanism for safer care. As one review puts it, “compassionate interactions… can help to identify and address potential risks… that could endanger patient safety”. If IPC guidance is to be truly fit for purpose and for all people, it must move beyond acknowledging person-centredness to embedding it as a core element of patient safety. That means supporting staff to communicate risk in ways that inform rather than alarm, creating space for proportionate discretion, and treating patients and families as partners in prevention rather than passive recipients of rules. IPC can be person-centred. Some are already leading the way. But there is some way to go for this to be fully considered the norm across our health care systems. So where do we go from here? When IPC practitioners were asked on a recent webinar what person-centred IPC looks like, the answers were strikingly human: compassion, empathy, dignity, connection. For those designing, leading and delivering healthcare, a simple question may therefore be useful: Does our guidance support not only the prevention of infection, but the experience of safe, compassionate care? If the answer is not yet fully yes, then there is an opportunity for improvement.